Method of remote collaboration system

ABSTRACT

The invention concerns using multiple computers to hold a conference. Under the invention, an application program can run on a single computer, yet remote participants can issue commands to the program. Remote participants can watch the program operate, because the invention replicates the display window of the running program onto the displays of the remote computers. Any participant can make annotations on the participant&#39;s own computer display. The invention copies the annotations to the displays of the other participants.

This application is a Continuation of application Ser. No. 08/828,605,filed Mar. 31, 1997, now U.S. Pat. No. 6,061,717 entitled “REMOTECOLLABORATION SYSTEM WITH ANNOTATION AND VIEWER CAPABILITIES”, whichapplication is incorporated herein by reference.”

The invention concerns systems which allow multiple users to remotelyoperate a single computer program. The invention generates a commonvisual image which is distributed to all computers. The users can makeannotations on the common display. When they do, the inventionreplicates the annotations on all displays. Annotations can be keptprivate by users, if desired.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to:

application Ser. No. 08/035,091, now U.S. Pat. No. 5,835,713, issuedJul. 15, 1998, entitled “Remote Collaboration System,” filed on samedate herewith by FitzPatrick et al. and assigned to the assignee of thisapplication;

application Ser. No. 08/811,078, now U.S. Pat. No. 5,944,785, issuedAug. 31, 1999, entitled “Remote Collaboration System,” filed on samedate herewith by Pommier et al. and assigned to the assignee of thisapplication; and

application Ser. No. 08/034,313, now U.S. Pat. No. 5,608,872, issuedJul. 15, 1997, entitled “Remote Collaboration System,” filed on samedate herewith by Schwartz et al. and assigned to the assignee of thisapplication.

REFERENCE TO A MICROFICHE APPENDIX

A microfiche appendix, containing 2 microfiche and 142 total frames isfiled herewith.

BACKGROUND OF THE INVENTION

Modern telephone systems allow multiple parties at different locationsto hold a conference. However, telephone conferences do not provide allof the conveniences of a face-to-face conference, where participants allmeet at a common table in a meeting room.

For example, in a meeting room, participants can view an object ofinterest, such as a drawing or a product. Such viewing is not possiblein a telephone conference.

The invention provides a system which duplicates many of theconveniences of a conference where people are physically present, butallows them to be at remote locations.

OBJECTS OF THE INVENTION

It is an object of the invention to provide an improved electronicconferencing system.

It is a further object of the invention to provide a system which allowsusers to remotely operate a computer program.

It is a further object of the invention to provide a system which allowsmultiple computers to operate a single program residing on one of thecomputers.

It is a further object of the invention to provide a system which allowsmultiple computer users to view and annotate a common display.

SUMMARY OF THE INVENTION

In one form of the invention, multiple computers are linked together. Asingle computer runs an application program. The display produced by theprogram is replicated on the displays of all other computers. The userof each computer can annotate the user's own display, and the inventionreplicates the annotations on all other displays. The invention allowsusers of other computers to run a program which is loaded on a differentcomputer.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates three computers, connected by telephone lines.

FIG. 2 initiates an example, which will be elaborated in FIGS. 3-14. Inthe example, a calculator program is operated, and annotated, by variousparties.

FIG. 3 illustrates how the invention responds when a host user attemptsto operate the calculator, when the invention is in Annotation mode.

FIG. 4 illustrates how the invention responds when a host user attemptsto annotate the calculator, when the invention is in Annotation mode.

FIG. 5 illustrates how the invention responds when a host user attemptsto operate the calculator, when the invention is in Application mode.

FIG. 6 illustrates how the invention responds when a host user attemptsto operate the calculator, when the invention is in Local Annotationmode.

FIG. 7 illustrates how the invention responds when a host user attemptsto annotate the calculator, when the invention is in Local Annotationmode.

FIG. 8 illustrates how the invention responds to a host user when inView mode.

FIG. 9 illustrates how the invention responds when a remote userattempts to operate the calculator, when the invention is in Annotationmode.

FIG. 10 illustrates how the invention responds when a remote userattempts to annotate the calculator, when the invention is in Annotationmode.

FIG. 11 illustrates how the invention responds when a remote userattempts to operate the calculator, when the invention is in Applicationmode.

FIG. 12 illustrates how the invention responds when a remote userattempts to operate the calculator, when the invention is in LocalAnnotation mode.

FIG. 13 illustrates how the invention responds when a remote userattempts to annotate the calculator, when the invention is in LocalAnnotation mode.

FIG. 14 illustrates how the invention responds to a remote user when inView mode.

FIGS. 15 and 15A illustrate logic flow used by the invention.

DETAILED DESCRIPTION OF THE INVENTION Overview

FIG. 1 shows three computers connected by telephone links. Each computerruns a message-driven, multi-tasking, Graphical User Interface (GUI),such as that sold under the name Windows, available from MicrosoftCorporation, located in Redmond, Wash. Such GUIs are also calledoperating environments.

The user of a GUI interacts with a program by way of windows. Theinvention replicates selected windows, rather than the entire display,at the remote computers. This selective replication allows users tomaintain private areas on their displays, which are not shared.

Each computer also runs software developed by the inventors. Inaddition, one computer (the Host) runs an Application program. (It ispossible for the Host to run both programs because of the multi-taskingcapabilities of the GUI.)

The invention has four basic modes of operation:

1. Application Mode

Any user of any of the three computers in FIG. 1 can issue commands tothe Application program. For example, assume the Application program isone which simulates a hand-held calculator. The initial situation isshown in FIG. 2, where each computer display shows the calculator.Assume that the following events occur:

The user of the Host presses the “3” button on the calculator (either bykeyboard input, or mouse input, depending upon the design of thecalculator program). In response, each calculator, in its display area,shows a “3”.

The user of one Remote presses “+”.

The user of the other Remote presses “6”.

The user of the Host presses “=”.

At this point, all calculators will display “9”, which is the sum of 3and 6. The users collectively operated the calculator program, and thedisplay of each shows the result.

The calculator program does not care which users pressed the buttons,nor whether some users pressed no buttons, provided a legal sequence ofbuttons was received. (It is assumed that the users are cooperative, andthat no users try to sabotage operation of the calculator.)

2. Annotation Mode

Any user can draw on the user's own, local, display, using drawing toolssimilar to those found in a “paint” program. The user can draw boxes,circles, arcs, text, ellipses, and so on. The user can also erase itemson the display.

The invention can replicate the user's annotations on all otherdisplays, so that all users view similar displays. However, the displayscould be different, because of the following factors:

(A) Different display monitors have different properties, such asresolution and color capability.

(B) Different display protocols (EGA, VGA, etc.) represent graphicsimages differently, and have different color capabilities.

(C) Different GUIs, or different versions of the same GUI, may havedifferent display conventions. Different computers in FIG. 1 could runthe different GUIs.

(D) Some users have changed the size of the window in which theircalculator is displayed, causing a deviation in scaling.

These differences can cause differences in the appearance of thedisplayed images, relative to each other, but the basic content of alldisplays should be the same. To accommodate size differences, theinvention draws to different scales as appropriate.

3. Local Annotation Mode

A user can annotate the local display, but the annotations are keptprivate, and no other user can see the annotations.

4. View Mode

No users can annotate, nor can they issue commands. However, an actionresembling annotation can be taken. Users can move their cursors, andothers will see the movement, allowing remote pointing. View Mode isuseful in one embodiment, wherein, for example, Annotate Mode is inforce, but a specific user's mode is designated as View. In thisembodiment, all users can annotate, but the “View” user can only watch,and cannot annotate.

Explanation of Individual Modes

FIGS. 3-14 will illustrate the different modes, by way of example, usingthe calculator program.

FIG. 3 Host Runs Application Program Mode is “Annotation” User Input isat Host Computer User Attempts to Operate Calculator

Assume that the user of the host computer attempts to add two numbers,using the calculator. Attempted entry of the first number will beconsidered.

The user, located at the Host, moves the Host's cursor over a key of thecalculator, as shown in FIG. 3, and tries to depress the key, byclicking the mouse. However, the mouse click does not reach theApplication program, because the invention blocks it. The Applicationprogram does not respond, because it receives no mouse click.

That is, in more detail, the GUI detects the mouse movement, and causes“mouse messages” to be generated. The GUI places the mouse messages intoa queue, where they await processing. INPUT ROUTER in FIG. 15 readsthese messages. Because “Annotation Mode” is currently in force, INPUTROUTER directs the messages to the ANNOTATION block. APPLICATION doesnot receive the messages, and thus does not respond. The mouse click isignored.

ANNOTATION's Response

ANNOTATION can be configured to respond in two (or more) ways to themouse messages. In one configuration, ANNOTATION requires the mouse toinitially select an ANNOTATION TOOL. If no selection is done, ANNOTATIONignores mouse messages.

Selection is done by clicking the mouse over an image of the tool, as iscommonly done in “paint” programs. ANNOTATION recognizes this toolselection, and then treats subsequent mouse clicks as data for drawingwith the selected tool. For example, if a rectangle tool were selected,the next two mouse clicks would define the diagonal corners of therectangle. (FIG. 4, later discussed, illustrates drawing a rectangle.)

Under the second configuration, a default tool, such as a pen, isautomatically selected when in Annotation Mode. In this configuration,when the user tries to depress a calculator button (by clicking on it),the user (unintentionally) initiates drawing of a line, using the pen.When the user recognizes this, the user can terminate drawing of theline, in any of several known ways.

Therefore, in Annotation Mode, the invention either (a) responds tomouse input by initiating a default annotation, or (b) ignores the mouseinput, because an annotation tool was not selected. Keyboard input fromthe user is treated the same way. Of course, other responses byANNOTATION can be designed.

Tracking of Cursors

Each display shows a cursor whose position is controlled by theassociated mouse. The invention replicates each cursor on all displays.Thus, in FIG. 3, with three mouses, there are three cursors on eachdisplay (only one is shown for simplicity).

Consequently, when one user moves a mouse, the corresponding cursormoves on all displays.

In general, the three cursors are distinguishable: each cursoridentifies its owner, as by color, shape, inclusion of a label, or thelike.

FIG. 4 Host Runs Application Program Mode is “Annotation” User Input isat Host Computer User Attempts to Draw a Box over the Calculator

This situation is quite similar to that of FIG. 3, except that, now, theuser intends to draw an annotation, instead of intending to press abutton, as in FIG. 3.

Assume that the user of the host computer draws a box over thecalculator. (The box is shown overly large, for emphasis. It ispreferred that the box not extend beyond the calculator itself.) Theinvention replicates the box on the remote computers. (The box is drawnusing annotation tools, which are not shown.)

In terms of FIG. 15, INPUT ROUTER directs the logic flow to ANNOTATION.ANNOTATION calls the proper GDI functions to draw the box. Also,ANNOTATION sends “annotation messages” to CONNECTION API, which deliversthe annotation messages to the Remotes.

ANNOTATION in FIG. 15A receives the annotation messages. This ANNOTATIONblock represents the logic executed at each remote computer. ThisANNOTATION calls the proper GDI functions, via the block GDI.

“GDI” is an acronym for Graphical Device Interface. “GDI functions” aresmall programs, contained in a larger program of the GUI called GDI.EXE.A GDI function, when called, draws a specific graphic image, such as acircle, box, or text, based on subsequent input from the user. Other GDIfunctions perform other tasks, such as selecting pen widths.

GDI.EXE is a commercially available product. Technical detailsconcerning GDI.EXE are contained in “Windows Software Development Kit,”available from Microsoft Corporation, and in Programming Windows 3.1 byCharles Petzold (Microsoft Press, Redmond, Wash., 1992, ISBN1-55615-395-3).

FIG. 5 Host Runs Application Program Mode is “Application” User Input isat Host Computer User Attempts to Use Calculator

The user of the Host moves the cursor over the calculator key “3” andclicks the mouse. The GUI generates a mouse message and places in intothe queue. The invention reads the mouse message, and passes the messageto the Application program (ie, the calculator program), which respondsby (1) showing that the key “3” is depressed and (2) drawing the numeral“3” in the calculator's display, using GDI calls. The Applicationprogram also records the fact that the user enters a “3,” for its owninternal operations.

The invention also intercepts the GDI calls made by the Applicationprogram in drawing the “3” in the calculator, and in drawing thedepressed “3” button. The invention notifies the other computers of theGDI calls. The other computers replicate the Host display, by executingthe same GDI functions. Greater detail concerning this GDI interceptionis given later, in the section entitled “General Considerations.”

Thus, all users simultaneously see the user of the Host operate thecalculator. (The action is not exactly simultaneous, because extremelyshort delays are involved. However, a human probably could not detectthe delays if the Host and the Remote were operating side-by-side.)

In terms of FIG. 15, the INPUT ROUTER recognizes that the mouse messagesshould be directed to the Application program, and directs the logicflow to APPLICATION (ie, the calculator program). APPLICATION (1) drawsa depressed “3” key and (2) writes the numeral “3” in the calculator'sdisplay, by calling appropriate GDI functions.

However, the invention, via GDI CAPTURE in FIG. 15, captures theApplication program's GDI calls, before they are executed. The inventiondoes two things with the captured calls. One, it notifies the othercomputers of these calls, via the block CONNECTION API. This actionleads to block CAPTURED GDI DISPLAY in FIG. 15A, which causes eachRemote to execute the same GDI functions, as indicated by block GDI.

Two, the invention allows the GDI functions, called by the Applicationprogram, to be executed at the host, via the block GDI in FIG. 15.

Therefore, the invention captures GDI function calls made by theApplication Program. The invention notifies the Remote computers of thecaptured calls, so that the Remotes can duplicate them. The inventionallows the captured calls to be executed as intended on the Host.

FIG. 6 Host Runs Application Program Mode is “Local Annotation” UserInput is at Host Computer User Attempts to Operate Calculator

Assume that in Annotation Mode, there is no default annotation toolgiven to the user. Under this assumption, if the user moves the cursorto a calculator button, and tries to “press” the button, the INPUTROUTER in FIG. 15 passes the mouse message to the ANNOTATION block.Since the mouse click is not part of a valid annotation input sequence(no tool was selected), ANNOTATION draws nothing.

Further, the Remote computers do not show the movement of the cursorcorresponding to the Host computer's mouse, as indicated, because line 5in FIG. 15 does not send Annotation Messages to the other computers whenLocal Annotation is in force.

Further still, the calculator button is not re-drawn as a depressedbutton on the Host display, in response to the attempt to press it,because APPLICATION did not receive the mouse message. APPLICATION isresponsible for drawing depressed calculator buttons.

If a default annotation is assigned to the user in Local AnnotationMode, the user's mouse click would initiate drawing by that tool. Whenthe user realized the mistake, the user would terminate the drawing, ina known manner.

FIG. 7 Host Runs Application Program Mode is “Local Annotation” UserInput is at Host Computer User Attempts to Annotate Calculator

Under these conditions, the INPUT ROUTER in FIG. 15 recognizes a validattempt to perform annotation, as by drawing a box. The INPUT ROUTERdirects the logic flow to the ANNOTATION block, which calls the properGDI functions for drawing the annotation, namely, a box, as shown inFIG. 7.

However, because the annotation is local, no boxes are drawn on remotecomputers, as indicated in FIG. 7. No data is sent along data path 5 inFIG. 15.

FIG. 8 Host Runs Application Program Mode is “View” User Input is atHost Computer User Attempts to Operate Calculator

As FIG. 8 indicates, the mouse click is ignored, and nothing happens atthe Remotes.

In FIG. 15, the INPUT ROUTER reads the mouse message, but blocks it fromAPPLICATION, because the current mode is “view.”

FIG. 9 Host Runs Application Program Mode is “ANNOTATION” User Input isat Remote Computer User Attempts to Operate Calculator

Assume that the user moves the mouse cursor over a calculator button andclicks the mouse. The mouse click is ignored. The other computers (Hostand the other Remote) show the motion of the user's cursor, but nothingelse, because no tool has been selected.

In FIG. 15A, the INPUT ROUTER blocks the mouse message from reachingAPPLICATION. The logic is directed to ANNOTATION, which draws a cursoron the user's Remote display, via block GDI. ANNOTATION also sends datato CONNECTION API, which directs the logic to ANNOTATION in FIG. 15.This ANNOTATION represents the annotation logic present on the two othercomputers: the Host and the other Remote. These ANNOTATION blocks drawcursors corresponding to the users cursor, at corresponding positions,via the GDI block in FIG. 15, which represents GDI function calls.

The Host can use one tool, such as a box-drawing tool, while a Remotecan use a different tool, such as a circle-drawing tool.

FIG. 10 Host Runs Application Program Mode is “ANNOTATION” User Input isat Remote Computer User Attempts to Annotate Calculator

Assume that the annotation is a box. A box is drawn on all displays. InFIG. 15A, the INPUT ROUTER at the user's Remote directs the mousemessages to the block ANNOTATION. ANNOTATION does two things. One, itcalls the proper GDI functions to perform the annotation, namely,drawing the box.

Two, ANNOTATION sends annotation messages to CONNECTION API, whichdelivers the annotation messages to the other computers. However, one ofthese is the Host, and the other is a Remote. The logic at the Hostreaches ANNOTATION in FIG. 15, and the logic at the other Remote reachesANNOTATION in FIG. 15A.

Both of these ANNOTATION blocks cause the proper GDI functions to becalled, to draw an annotation corresponding to the user's annotation.However, in the Host, logic path 5 is not taken at this time, because itis not necessary to replicate the Host's annotations at other computers.

FIG. 11 Host Runs Application Program Mode is “APPLICATION” User Inputis at Remote Computer User Attempts to Operate Calculator

The reader is reminded that the calculator program is loaded only on thehost, while a Remote user wishes to operate it.

The Remote user's INPUT ROUTER in FIG. 15A routes the mouse messages toCONNECTION API. The Host receives these messages, which are delivered tothe Host's INPUT ROUTER in FIG. 15. The Host's INPUT ROUTER directs themessages to the block APPLICATION (ie, to the Application program,namely, the calculator program), which does two important things.

The calculator program treats the messages as though they were issued bythe Host's mouse, even though a Remote mouse caused them. The calculatorprogram responds in its usual way, which includes (1) showing adepressed calculator button “3”, (2) writing the numeral “3” in thecalculator's display, and (3) performing its own internal computationswhen it learns that the user entered data (namely, the “3”).

However, before the calculator program can execute (1) and (2) in theprevious paragraph, the Invention first captures the GDI functions whichthe calculator program calls. This capture is illustrated in block GDICAPTURE in FIG. 15.

During this capture, the Invention, in effect, does two things. One, itsends these GDI functions to CONNECTION API (for the other computers touse). At the user's Remote, CONNECTION API in FIG. 15A directs the GDIfunctions to CAPTURED GDI DISPLAY, which replicates the Host's display.Two, it causes the GDI functions to be executed at the Host (via blockGDI in FIG. 15). Therefore, the general sequence of events is thefollowing:

The Remote user attempts to press a calculator button.

The invention running on the Remote detects this attempt, and sends datato the calculator program running on the host. The data takes the formof messages, which the calculator program “thinks” come from the Host'smouse.

The calculator program performs as usual, and draws images on the Hostdisplay, via GDI calls.

The invention captures the GDI calls, and informs the Remotes of them.

The Remotes replicate the Host's window. The Remote user thus canremotely operate the calculator program running on the Host.

Summarizing in a different way: The invention generates mouse messagesat the Host, based on mouse messages at the Remote. The calculatorprogram (running on the Host) responds to the mouse messages as thoughthey were generated at the Host. The invention intercepts the GDI callsmade by the calculator program, and executes the same GDI calls at theRemote, thereby replicating the Host's display at the Remote.

FIG. 12 Host Runs Application Program Mode is “Local Annotation” UserInput is at Remote Computer User Attempts to Operate Calculator

The user's mouse click is ignored. Nothing appears on the other displaysin response to the mouse movement, because of failure to select a tool.

FIG. 13 Host Runs Application Program Mode is “Local Annotation” UserInput is at Remote Computer User Attempts to Annotate Calculator

The annotation is drawn on the user's display, as indicated. Noannotation occurs on the other displays.

FIG. 14 Host Runs Application Program Mode is “View” User Input is atRemote Computer User Attempts to Operate Calculator

As indicated, the mouse cursor moves at the user's display, but themouse click is ignored. Further, the other two displays do not show themovement of the user's mouse cursor.

General Considerations

1. Different Programs Draw Different Parts of Overall Display.

The displays are drawn using GDI functions. However, different parts ofa display are drawn by different programs.

Despite the fact that all these drawing operations are undertaken usingGDI functions, GDI functions are not the exclusive medium ofcommunication between computers for replicating the displays.

Annotation Involves One Type of Data Transfer Among Computers

Drawing by an Application Program Involves Another Type.

For example, when a user performs annotation, the user's mouse messagesare replicated, AS MESSAGES, at the other computers, via path 5 in FIG.15. These replicated messages then cause the respective ANNOTATIONblocks (at the other computers) to issue the proper GDI calls fordrawing the annotation. That is, GDI calls are not sent directly fromthe user performing the annotation to the other computers.

In contrast, when an application program causes a graphic image to bedrawn on a display, the invention intercepts GDI calls (via GDI CAPTUREin FIG. 15) and causes the GDI calls to be replicated on the othercomputers.

Reason for Difference

A major reason for the two different procedures (replicating mousemessages and replicating GDI calls) is that annotations are stored inmemory at different locations than the display information.

That is, returning to the calculator of FIG. 2, the Application programstores the image of the calculator in the following general way.Annotation data is stored by the invention; Application program data isstored by the Application program (at the host). Each image of a key isstored as data from which a GDI function can draw the key. The dataincludes information such as position, size, color, and so on. Each keyincludes an associated number. The number can be stored as a textcharacter, with information as to position, size, font type, and so on.

Annotation data is stored at a different location, but in the samegeneral way.

If either the annotation or the Application program needs bitmaps, thebitmaps are stored in a conventional, known manner, by the GUI.

The invention combines the annotation images with the Application'simages by the known technique of masking. That is, the invention, at aRemote, plays (or executes) the received GDI functions into a bitmap.The invention plays the received annotation information into a differentbitmap. The two bitmaps are masked together.

The annotation data is kept separate from the application data so that,for example, a user can save an Application image, but withoutannotations. Alternately, a user can save annotation data alone, or savean annotated display.

As another example, keeping the annotation data separate facilitatesdrawing a display having no annotation data. If the annotation data wereintermingled with the calculator image data, elimination of theannotation data would be difficult, if not impossible.

If GDI calls were transmitted exclusively (ie, no message replicationwere undertaken), then extra effort would be required to constructannotation data for separate storage.

2. GDI Interception, or Capture.

GDI interception can be understood as follows.

A. On start-up, the invention replaces the first five bytes of each GDIfunction with a JUMP instruction to a particular program, namely,Trap.GDI.

B. Trap.GDI gets the parameters for the desired graphics image (eg, inthe case of a box, the locations of the two diagonal corners) and callsthe sub-program PkgDispCall. Trap.GDI also replaces the first fivebytes.

C. PkgDispCall accepts the parameters from Trap.GDI and generates anobject structure. This object structure is a block of data containingeverything necessary for the other computers to draw the box.

For example, the object structure contains information as to size andposition of the box. Further, the GUI draws images within a “context.”The context includes things such as pen width, color, and otherfeatures. The invention tracks the contexts of the individual computers.If the context of the box drawn is different from the contexts of theremote computers, PkgDispCall includes data necessary for the othercomputers to create the correct contexts.

D. The object structure is shipped to the other computers, which thenexecute the same GDI functions.

E. The invention executes the original GDI functions.

3. Displays are not Transferred in Entirety.

The displays are not replicated bit-by-bit. For example, the image ofthe calculator in FIG. 2 could be transferred between computers inbitwise fashion. If the calculator occupied a space of 200×300 pixels,then information regarding 60,000 (ie, 200×300) pixels must be sent.

Instead, the particular calculator image shown in FIG. 2 is treated aseighteen rectangles, plus a text character for each of sixteen of therectangles, giving a total of 34 objects. Each object requiresparameters, such as size and position. The number of parameters issmall, in the range of three to ten. Assuming ten parameters, then 340pieces of data must be sent. Of course, the size of each piece dependson many factors, but a small number of bytes for each piece may beassumed.

Therefore, the invention reduces the 60,000 pieces of data needed forbitwise replication to 340 pieces maximum for object replication. Ofcourse, some objects may take the form of bitmaps, and must be sentbit-by-bit. However, in general, bitmaps are expected to be rare.Further, it is expected that, in general, bitmaps, when sent, need besend only once.

Further, the object data is compressed when possible. That is, everytransmission between computers is of compressed data, when possible.Compression is known in the art.

4. Types of Data Link.

Communication among computers can take several forms. Commerciallyavailable networks, local and wide area, can be used. Commerciallyavailable ISDN telephone service, provided by local telephone companies,can be used. Modem communication can be used.

5. Prior Art Message Detection.

There are commercially available packages which detect messagesgenerated by the GUI in response to an input device. One such package isWINSIGHT, available from Borland International. However, it is believedthat such packages do not inform remote computers of the messages.

6. Alternate GDI Capture.

An alternate approach to the graphics capture described above is thefollowing. The system-provided GDI is replaced by a separate procedurewhich processes GDI calls before calling the actual system GDI. Thesystem GDI name is changed to prevent confusion between the two modules.The same technique is also used on USR.EXE to also capture GDI callsmade through system-provided modules.

7. More than One Computer can Run Application Programs.

A given computer can act as a Host for one program and a Remote foranother. For example, one computer can run a word processing program.Another computer can run a CAD drawing program. Each is Host for itsrespective program.

Since the invention's software on each computer is identical, orsubstantially identical, all users can run either the word processingprogram or the CAD program, in the manner described above.

8. “Real” Cursors and “Pseudo” Cursors.

There are two types of “cursor. ” Each GUI generates its own “real”cursor. The real cursor is not generated by GDI functions, but by anindependent function in the GUI. The reader can view the cursor as abitmap which the GUI moves in response to mouse motion.

In addition to the real cursor, which is controlled by the local mouse,the invention generates a “pseudo” cursor for each remote participant.The pseudo cursors are generated using GDI functions.

Sometimes a real cursor changes shape as the cursor moves. For example,it can take the form of an arrow when lying on a tool bar, and thenchange to a hand when lying on a client area. Sometimes this change isunder the control of the Application program.

Therefore, if a Remote user is controlling an Application programrunning on a Host machine (as in FIG. 11), the Application program maychange the cursor on the Host machine, but without using GDI calls.Consequently, the GDI capture of FIGS. 15 and 15A will be ineffective toreplicate the changed on the Remote display.

To confront this problem, the invention watches for the functions whichchange the real cursor (eg, the SetCursor command). The inventionreplicates the cursor change on the Remote computer.

One way is to execute the same SetCursor command. An alternate approachwould be to change the Remote cursor by executing a proper sequence ofGDI calls, or to draw a bitmap, when the Host cursor changes.

9. Entire Display not Replicated.

The invention only replicates windows which the user of a displayidentifies. That is, the user can keep a workspace, such as a notepad,private during a conference. GDI calls use a task handle. If the taskhandle does not refer to a shared item, the GDI calls are not shared.

10. Computer Code.

Computer code in microfiche form is attached. A description of filescontained therein is contained in the following Table.

What is claimed is:
 1. A method for allowing multiple parties tocollaborate, comprising: a) running a program that is stored on a hostcomputer; b) sharing the running program with at least one remotecomputer; c) linking each remote computer and the host computer togetherusing a network; d) running substantially identical programs on eachremote computer, wherein running the program on each remote computerallows a user of each of the remote computers to perform the stepsof: 1) selecting and running a program stored in the storage space ofthat remote computer; 2) providing input to the selected program; 3)showing an output of the selected program selected on the display of atleast one of the remote computers, 4) allowing the user of each remotecomputer to draw at least one annotation image on the display of thatuser's remote computer; and 5) replicating the annotation images on thedisplays of the computers, wherein one or more of the steps of d)1)through d)5) are accomplished by a computer program which is distinctfrom the shared program.
 2. A method for allowing multiple parties tocollaborate, comprising: a) using a plurality of user computers, eachhaving a display and storage space for programs; b) running a computerprogram stored at a host computer other than the plurality of the usercomputers, wherein the computer program is shared by the host computerand the plurality of user computers; c) linking the host computer andthe plurality of user computers together using a network; d) runningsubstantially identical. programs on each of the user computers; e)selecting and running a program stored in the storage space of one ofthe user computers through use of the substantially identical programs;f) providing program input to the program selected through use of thesubstantially identical programs at at least one of the user computers;g) showing output of the program selected on the display of at least oneof the user computers; h) allowing a user of each of the user computersto draw annotation images on the display of that user's computer; and i)replicating the annotation images on the displays of at least one otherof the user computers.
 3. A method for exchanging information viacomputers, comprising: a) using a plurality of user computers, eachhaving a display and storage space for programs; b) running a computerprogram stored at a host computer, wherein the computer program isshared by the host computer and the plurality of user computers; c)linking the host computer and the plurality of user computers togetherusing a network; d) running substantially identical programs on each ofthe computers; e) selecting and running a program stored in the storagespace of one of the computers through use of the substantially identicalprograms; f) providing program input to the program selected through useof the substantially identical programs at at least one of the usercomputers; g) showing output of the program selected on the display ofat least one of the computers; h) allowing a user of each of the usercomputers to draw annotation images on the display of that user'scomputer; i) replicating the annotation images on the display of atleast one other of the user computers; j) having selected ones of theuser computers as annotators; and k) preventing the selected programfrom responding to program input provided by annotators.